Identifying Portions of Electronic Communication Documents Using Machine Vision

ABSTRACT

In a computer-implemented method, an artificial neural network is trained to identify portions of conversation segments within electronic communication documents, wherein an input layer of the artificial neural network includes a plurality of input parameters each corresponding to a different characteristic of text-based content. The method also includes receiving a first electronic communication document that includes first text-based content, and processing the first text-based content using the trained artificial neural network. Processing the first text-based content includes generating one or more position indicators for the first electronic communication document, and the one or more position indicators include one or more segment portion indicators denoting positions of one or more portions of one or more conversation segments within the first electronic communication document. The method also includes determining an ordered relationship between the first electronic communication document and one or more other electronic communication documents using the position indicator(s).

CROSS-REFERENCE TO RELATED APPLICATIONS

This is a continuation of U.S. patent application Ser. No. 15/612,658,entitled “Identifying Portions of Electronic Communication DocumentsUsing Machine Vision” and filed on Jun. 2, 2017, the entirety of whichis hereby incorporated herein by reference.

FIELD OF THE DISCLOSURE

The present disclosure generally relates to electronic document reviewand, more specifically, to techniques for processing electroniccommunication documents (e.g., emails) prior to user review.

BACKGROUND

In various applications, a need exists to extract meaningful informationfrom a corpus of electronic documents. In the discovery process commonlyassociated with litigation, for example, attorneys are commonly provideda large corpus of electronic documents, including electroniccommunication documents (e.g., emails) that were received from, or maybe sent to, an opposing party. Given the potentially enormous number ofsuch documents (e.g., in the millions), analyzing each and everyelectronic communication document can be an extremely time-consumingprocess. Typically, many of these electronic communication documentsconvey redundant information. In an email context, for example, thecorpus of emails may include a copy of a particular email from thesender's outbox, and another copy from the inbox of each recipient. Insuch instances, a reviewer does not need to review each copy of theemail to determine whether the email is relevant to the discoveryprocess. As another example, an email message may include informationfrom previous emails within an email chain (e.g., as can be seen byscrolling down while viewing the email), with the final email of a chaintypically containing all of the information conveyed by prior emailswithin the same “conversation.” In such instances, these prior emailscan be safely discarded or ignored without losing any meaningfulinformation.

“Threading” (e.g., “email threading”) is a process that reduces thenumber of documents in a corpus of electronic communication documents byremoving electronic communication documents that fail (or very likelyfail) to convey new information. An email may convey new information,if, for example, the email includes a new recipient or attachment, thesubject and/or the body of the email is not included in any other emailsin the same chain or conversation, and/or the email is a final email inthe chain or conversation. Electronic document review tools thatorganize electronic communication documents according to thread canprovide great efficiencies in the user review process. For example, auser reviewing documents may be able to quickly identify which emailswithin a particular corpus of emails share a common thread (or share acommon group of related threads that branch off of each other), andfocus solely on that set of emails before moving on to the next threador thread group.

To arrange electronic communication documents into conversation threads,the documents are generally pre-processed (i.e., processed prior to userreview of the documents) to generate metadata indicating the orderedrelationship among the documents within each thread. In one techniquefor determining such ordered relationships, the threading processrequires identifying a number of different “conversation segments” ineach document, where each conversation segment corresponds to a singlecommunication from a single person. In a given email, for example,earlier conversation segments can usually be seen by scrolling down tolook at previous messages in the same email chain, with each segmentincluding a header, a message body, and possibly a signature block. Theordered relationships may then be determined by comparing theconversation segments (or segment portions) of one electroniccommunication document to the conversation segments (or segmentportions) of other electronic communication documents, with any matchingsegments or segment portions generally indicating that two differentdocuments belong to the same thread or the same thread group (i.e., aset of threads all sharing the same root document).

Unfortunately, various issues can make it difficult to accuratelyreconstruct a thread. Accurate thread reconstruction typically requiresaccurate identification of conversation segments, segment sections(e.g., headers), and/or segment fields (e.g., header fields such assender, recipient, and/or date/time). The task of identifying segments,segment sections, and/or segment fields can be complicated by the factthat, at least in certain scenarios, characteristics that mightotherwise be reliable indicators of these elements can vary. Forexample, different software clients (e.g., Microsoft Outlook, LotusNotes, etc.) may use different names/keywords for the same field (e.g.,“From:” versus “Sender:” or “By:” or “Author:”), and/or may place thesame field at different positions within a header or other segmentsection. Moreover, the field keywords and/or positions may vary overtime even for a single software client, as new versions of the clientcome into widespread use. As a result, software developers for threadingtools may need to continually play “catch up” as new formats (e.g., newheader formats) appear, by writing code that is able to properly parsefields according to each new format. Furthermore, certain field keywordsmay change even among subsequent conversation segments within a singleelectronic communication document, depending on the language of thesender and/or recipient at each stage of the conversation.

Typically, if even a single conversation segment or segment portion(e.g., header field) of a particular electronic communication documentcannot be identified, the entire document is discarded or ignored forpurposes of thread reconstruction. Thus, the above-noted difficultiesassociated with conventional parsing of electronic communicationdocuments can lead to a significant loss of information for reviewingusers.

BRIEF SUMMARY

In one aspect, a computer-implemented method computer-implemented methodfor identifying portions of electronic communication documents includes:(1) training, by one or more processors analyzing a test set ofelectronic communication documents, an artificial neural network toidentify portions of conversation segments, within electroniccommunication documents, wherein the artificial neural network includesa plurality of layers, and wherein an input layer of the plurality oflayers includes a plurality of input parameters each corresponding to adifferent characteristic of text-based content; (2) receiving, by theone or more processors, a first electronic communication document thatincludes first text-based content; (3) processing, by the one or moreprocessors, the first text-based content of the first electroniccommunication document using the trained artificial neural network,wherein processing the first text-based content includes generating,within the plurality of layers, one or more position indicators for thefirst electronic communication document, and wherein the one or moreposition indicators include one or more segment portion indicatorsdenoting positions of one or more portions of one or more conversationsegments within the first electronic communication document; and (4)determining, by the one or more processors, an ordered relationshipbetween the first electronic communication document and one or moreother electronic communication documents using the one or more positionindicators for the first electronic communication document.

In another aspect, a computing system includes an electronic documentdatabase, one or more processors, and one or more memories. The one ormore memories store instructions that, when executed by the one or moreprocessors, cause the computing system to: (1) train, by analyzing atest set of electronic communication documents, an artificial neuralnetwork to identify portions of conversation segments, within electroniccommunication documents, wherein the artificial neural network includesa plurality of layers, and wherein an input layer of the plurality oflayers includes a plurality of input parameters each corresponding to adifferent characteristic of text-based content; (2) retrieve, from theelectronic document database, a first electronic communication documentthat includes first text-based content; (3) process the first text-basedcontent of the first electronic communication document using the trainedartificial neural network, wherein processing the first text-basedcontent includes generating, within the plurality of layers, one or moreposition indicators for the first electronic communication document, andwherein the one or more position indicators include one or more segmentportion indicators denoting positions of one or more portions of one ormore conversation segments within the first electronic communicationdocument; and (4) determine an ordered relationship between the firstelectronic communication document and one or more other electroniccommunication documents using the one or more position indicators forthe first electronic communication document.

In another aspect, a non-transitory, computer-readable medium storesinstructions that, when executed by one or more processors, cause theone or more processors to: (1) train, by analyzing a test set ofelectronic communication documents, an artificial neural network toidentify portions of conversation segments, within electroniccommunication documents, wherein the artificial neural network includesa plurality of layers, and wherein an input layer of the plurality oflayers includes a plurality of input parameters each corresponding to adifferent characteristic of text-based content; (2) receive a firstelectronic communication document that includes first text-basedcontent; (3) process the first text-based content of the firstelectronic communication document using the trained artificial neuralnetwork, wherein processing the first text-based content includesgenerating, within the plurality of layers, one or more positionindicators for the first electronic communication document, and whereinthe one or more position indicators include one or more segment portionindicators denoting positions of one or more portions of one or moreconversation segments within the first electronic communicationdocument; and (4) determine an ordered relationship between the firstelectronic communication document and one or more other electroniccommunication documents using the one or more position indicators forthe first electronic communication document.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 depicts an example environment in which techniques for accuratelyreconstructing electronic communication document threads may beimplemented, according to one embodiment.

FIG. 2 depicts an example environment in which electronic communicationdocument threads may be accurately reconstructed and presented to auser, according to one embodiment.

FIG. 3 depicts an example artificial neural network that may be trainedby the neural network unit of FIG. 1 or the neural network unit of FIG.2, according to one embodiment and scenario.

FIG. 4 depicts an example neuron that may be included in the artificialneural network of FIG. 3, according to one embodiment and scenario.

FIG. 5 depicts text-based content of an electronic communicationdocument that may be processed by an artificial neural network of thepresent invention, according to one embodiment and scenario.

FIG. 6 is a flow diagram of an example method for identifying portionsof electronic communication documents, according to one embodiment.

FIG. 7 depicts an example user interface that may be used toautomatically generate header definition data, according to oneembodiment and scenario.

FIG. 8 is a flow diagram of an example method for facilitatingrecognition of header fields in electronic communication documents,according to one embodiment.

DETAILED DESCRIPTION I. Overview

The embodiments described herein relate to, inter alia, the processingof electronic communication documents (e.g., emails) to reconstructconversation threads. The systems and techniques described herein may beused, for example, in connection with electronic document review toolsof the sort commonly used during litigation. However, other applicationsare also possible. For example, the systems and techniques describedherein may be used by a company or other entity to categorize and/orreview its own archived electronic communication documents (e.g., emailsto and from customers, etc.), and/or for other purposes.

More specifically, the embodiments described herein relate to threadingtechniques that rely on the identification of conversation segmentswithin a given electronic communication document, and/or theidentification of portions of such segments (e.g., headers, and/orspecific header fields such as sender, recipient, and/or date, etc.).The term “conversation segment” (or simply “segment”), as used herein,generally refers to the incremental content (e.g., header, message body,and possibly signature block) added at each step of a communicationchain/thread, not including any modifications that may have been made toearlier segments of the conversation (e.g., by adding in-line responsesto an earlier email in an email chain). Thus, for example, a root/firstelectronic communication document generally includes only a singleconversation segment, a reply thereto generally includes exactly twoconversation segments, and so on, with each new reply or forward (ordraft thereof) adding an additional segment. In at least someembodiments (e.g., for email documents), each successive document withinthe chain/thread will typically contain both the most recentconversation segment and every previous segment, such that a reader canreference earlier stages of the conversation by looking further down inthe text of the document.

In some embodiments, identification of particular conversation segmentsand/or segment portions is accomplished using a “machine vision”technique that, in some respects, may mimic the process by which aperson consciously or subconsciously identifies segments and/or segmentportions within a communication document, even if that person isunfamiliar with the specific formatting of those segments and/or segmentportions. For example, the machine vision technique may identify headersby analyzing the spacing between lines, the length of lines, thefrequency and placement of certain delimiters (e.g., colons), thepresence of two and/or four digit numbers (e.g., indicating time and/ordate information), and so on. As another example, the machine visiontechnique may identify an author or sender field of a header byanalyzing whether a first word of a line within an identified header isfollowed by a colon, whether the colon is followed by two to four wordseach having only the first letter capitalized (e.g., a person's name),whether the line occurs prior to another, similar type of line (e.g.,corresponding to the recipient of the document), and so on.

In some embodiments, despite mimicking (to some extent) the process bywhich a human user might visually process a document, the machine visiontechniques described herein do not require processing any image files(e.g., raster image files such as JPEG, GIF, or PNG, or vector imagefiles). For example, the machine vision techniques described herein mayanalyze only (or primarily) non-image, text-based data of eachelectronic communication document, such as alphanumeric characters andassociated formatting or control elements (e.g., HTML or ASCII elementsrepresenting tabs, line breaks, etc.). Accordingly, the machine visiontechniques described herein represent a novel approach that utilizesnon-image-based processing to achieve results more akin to thosetraditionally associated with image processing.

In some embodiments, the machine vision system may make use of anartificial neural network, and train the neural network using a set ofelectronic communication documents having known characteristics (e.g.,known segment and header delineations, header field locations, etc.).The neural network may employ multiple layers of nodes or “neurons,”with each neuron determining a particular parameter (e.g., making aparticular decision), and with each layer corresponding to a differentlevel of granularity. For example, the neurons of a first layer of theneural network may examine an electronic communication document as awhole to identify conversation segments within the document, the neuronsof a second layer may examine those segments (using information from thefirst layer) to identify segment sections such as headers and/orsignature blocks, and the neurons of a third layer may examine thosesegment sections (using information from the second layer) to identifyfields such as sender, recipient, and date/time. In some embodiments, arecurrent neural network is used, with decisions made by neurons of onelayer being fed back to the previous layer. In this manner, therecurrent neural network may learn the importance of various parameters(i.e., the importance of particular document characteristics, and/or ofparticular parameters derived from such characteristics) to the variousdecisions made by the neurons of the machine vision system, and mayweigh the parameters accordingly for future document processing.

In some embodiments, users (e.g., users of an electronic document reviewtool) may provide information that helps tailor the system to a specificset of electronic communication documents. For example, users mayprovide their own electronic communication documents, along withindicators of known segment delineations, header locations, and/or othercharacteristics, as additional training documents for a neural network.This may aid a machine vision system by allowing the system to learnfrom documents arranged in a format that was previously unknown to thesystem, and/or by biasing the system towards the types of documents thatare more likely to be found in a particular set of documents.Alternatively (e.g., if machine vision is not employed for thepre-processing of documents, or to supplement a machine vision system),a mechanism may be provided whereby users submit header definition data(e.g., rules or regular expressions) that facilitates the parsing ofparticular header fields for a particular header format (e.g., a formatunfamiliar to the computing system that performs threading-relatedoperations). In some embodiments where user information is input to thesystem, an electronic document review tool provides a user interfacethat guides the user through a process of selecting various segments,segment portions, and/or segment fields, and automatically generatestruth data/labels for training (if used in a machine vision context) orheader definition data (if used in a system that parses headers in amore conventional manner) based on the user selections.

By replacing conventional threading techniques with one or more of thetechniques described herein, various advantages may be obtained. Forexample, even if a particular software client (e.g., Microsoft Outlookor Lotus Notes) implements a new version that changes a header (and/orother segment section) format, and even if electronic communicationdocuments generated by a particular software client of software clientversion are encountered for the first time, there may be no need todevelop new code to identify segments (and segment sections, fields,etc.) associated with those software clients and/or versions. As anotherexample, the techniques described herein may enable more accurateidentification of particular header fields in instances where thelanguage of header field names/keywords can change from document todocument or segment to segment.

The machine vision systems and techniques described herein may provideother advantages as well. For example, the use of a neural network withmultiple layers of granularity may allow an email or other electroniccommunication document to be added to a thread even if the machinevision system fails to identify certain segment sections, or certainfields, etc. This may provide a user with more information about thestructure of a conversation, as compared to conventional threadingtechniques that simply discard or ignore documents that cannot be fullyprocessed under a set of software client-specific (and/orversion-specific) parsing rules.

II. Example Environments for Reconstructing And/or Presenting ElectronicCommunication Document Threads

FIG. 1 depicts an example environment 10 in which a corpus of electroniccommunication documents 12 is staged for analysis via a content analysisplatform 14, according to one embodiment. Communication corpus 12 mayinclude a plurality (e.g., thousands, millions, etc.) of electroniccommunication documents. As used herein, the term “electroniccommunication document” generally refers to an electronic document thatrepresents an exchange (or a potential/planned exchange, as in the caseof a draft email) between two or more individuals. However, the term canalso (in some embodiments and/or scenarios) include documents that areaddressed from an individual to himself or herself (e.g., an email sentfrom the individual's personal email account to his or her work emailaccount). While some of the examples described herein refer specificallyto email, it should be appreciated that the techniques described hereinare applicable to other types of electronic communication documents. Forexample, some instant messaging applications may archive a conversationupon its conclusion. The electronic file that represents the instantmessaging conversation may be considered an “electronic communicationdocument.” As another example, social media platforms may support theirown form of messaging (e.g., a Facebook message, an Instagram directmessage, etc.). Each of these messages may also be considered an“electronic communication document.” Furthermore, recent email platformslike Slack blend several types of electronic communications into asingle conversation. Electronic files that underlie these types of emailplatforms may also be considered “electronic communication documents.”

Communication corpus 12 may be ingested into a staging platform 16 toorganize communication corpus 12 in a manner that facilitates efficientanalysis via content analysis platform 14. Communication corpus 12 maybe ingested into staging platform 16 by executing a computer program ona computing device that has access to the environment 10. The ingestionprocess may involve the computer program providing an instruction tostaging platform 16 as to a location at which communication corpus 12 isstored, for example. Using this location, staging platform 16 may accesscommunication corpus 12 for performing conversation threadingtechniques.

Staging platform 16 may analyze communication corpus 12 to arrange theelectronic communication documents into threaded conversations 22-1through 22-N, where N is any positive integer. As used herein, a“conversation thread” (or simply “thread”) refers to an ordered sequenceof electronic communication documents, starting at a first (“root”)document and proceeding to a single, final document, with eachsuccessive document in the thread corresponding to a particular useraction that was taken in connection with the immediately precedingdocument. Thus, for example, a single conversation thread may include aninitial email, a “reply-all” to the initial email (i.e., a reply to thesender and all other recipients of the initial email), a forward of the“reply-all” email, and a reply to the forwarded email. Each of threadedconversations 22-1 through 22-N may represent documents of only a single(non-branching) conversation thread, or may represent documents of agroup of multiple conversation threads that all have different endpoints(final documents) but share the same root electronic communicationdocument.

In the embodiment of FIG. 1, staging platform 16 includes a threadingunit 24 to generate threaded conversations 22-1 through 22-N (or, moreprecisely, data indicative of the ordered arrangements/relationshipswithin each of threaded conversations 22-1 through 22-N). This may beaccomplished in various different ways, depending on the embodiment. Forexample, threading unit 24 may generate a “fingerprint” for eachconversation segment of each electronic communication document. Thefingerprint may be a hash of one or more header fields (e.g., sender anddate/time) within each conversation segment, e.g., as discussed in U.S.patent application Ser. No. 15/205,980, filed on Jul. 8, 2016 andentitled “System and Method for Fingerprinting-Based ConversationThreading,” the disclosure of which is hereby incorporated herein byreference in its entirety. Threading unit 24 may compare the sets ofsegment fingerprints for different documents in order to identifymatching segments, which may in turn enable threading unit 24 toidentify which documents belong to the same thread, as well as theorder/arrangement of documents within the thread.

More generally, threading unit 24 may rely on information relating tospecific conversation segments within electronic communicationdocuments, and/or information relating to specific segment portions(e.g., header locations, header field values, etc.), to generatethreaded conversations 22-1 through 22-N. To provide such information,staging platform 16 may include a neural network unit 26. Neural networkunit 26 may identify locations of specific conversation segments withina given electronic communication document, locations of specific segmentsections (e.g., header, message body, and/or signature block) withindifferent segments, and/or locations of specific fields (e.g., headerfields) within different segments, depending on the embodiment. Inembodiments where field locations are identified, neural network unit 26may also use those locations to determine the corresponding fieldvalues. In the “fingerprint” embodiment discussed above, for example,neural network unit 26 may attempt to determine the values of a senderfield and a date/time field within each conversation segment. In someembodiments, determining a field value may involve identifying thelocation of an individual segment, identifying a section (e.g., header)within that segment, and then identifying the appropriate field withinthat section. In other embodiments, the field value may be determinedusing a more direct approach, such as identifying a particular headerfield without first identifying a particular conversation segment orheader.

To identify segments and/or segment portions of an electroniccommunication document, neural network unit 26 utilizes an artificialneural network (also referred to herein simply as a “neural network”).The neural network may be any suitable type of neural network, such as arecurrent neural network or a feedforward neural network, for example.The neural network may include a number (e.g., hundreds or thousands) ofnodes or “neurons” arranged in multiple layers, with each neuronprocessing one or more inputs to generate a decision or other output.Some example neural networks are discussed further below in connectionwith FIGS. 3 and 4.

To train the neural network to identify segments, segment portions,etc., a training unit 30 of neural network unit 26 may access electroniccommunication documents within a test corpus 32. Test corpus 32 containsa number (e.g., tens, hundreds, thousands, etc.) of electroniccommunication documents with known/labeled data (i.e., “truth” data).For example, test corpus 32 may include metadata indicating knowndelineations between conversation segments of electronic communicationdocuments, known delineations between segments sections (e.g., headers,message bodies, and/or signature blocks), and/or known locations ofspecific types of fields (e.g., the sender of the document, therecipient of the document, the date/time, etc.). The metadata may havebeen generated directly by a system developer, for example, orautomatically generated based on labels or other indications that wereadded or made by a system developer, customer, or other individual(e.g., as discussed further below in connection with FIG. 7, or inanother suitable manner).

Once the neural network has been trained, new document processing unit34 may apply the trained neural network to electronic communicationdocuments within communication corpus 12. The outputs of the neuralnetwork, indicating segment and/or segment portion locations, may thenbe provided to threading unit 24. Alternatively, or additionally, newdocument processing unit 34 may use field location information output bythe neural network to determine the values of particular fields, andprovide those field values to threading unit 24. Threading unit 24 maythen use the data from new document processing unit 34 to arrange theelectronic communication documents into threaded conversations 22-1through 22-N as discussed above.

Once generated, threaded conversations 22-1 through 22-N may be ingestedinto content analysis platform 14. In some embodiments, content analysisplatform 14 includes an electronic document review (EDR) interface thatenables one or more reviewers to analyze the threaded conversations 22-1through 22-N. In some embodiments, content analysis platform 14additionally, or alternatively, includes a conceptual indexing tool thatperforms clustering and/or other operations on the threadedconversations 22-1 through 22-N to assist the reviewer.

FIG. 2 depicts an example environment 100 that may correspond to oneembodiment of the environment 10 of FIG. 1, but also includes varioususer/client-side components. The environment 100 includes a clientdevice 102, a web server 104, and a staging server 106. Client device102 is communicatively coupled to web server 104 via a network 110.Network 110 may be a single communication network, or may includemultiple communication networks of one or more types (e.g., one or morewired and/or wireless local area networks (LANs), and/or one or morewired and/or wireless wide area networks (WANs) such as the Internet).Web server 104 may be remote from or co-located with staging server 106.Web server 104 and staging server 106 may each be an individual server,or may each include a group of multiple servers. Alternatively, webserver 104 and staging server 106 may be combined in a single server.

Generally, web server 104 hosts web services relating to electronicdocument review, which may be accessed/utilized by client device 102,and staging server 106 implements certain back-end operations (e.g.,conversation threading) in support of the document review servicesprovided to client device 102. For example, staging server 106 may beused as (or within) staging platform 16 of FIG. 1, and web server 104may be used as (or within) content analysis platform 14 of FIG. 1. WhileFIG. 1 shows only a single client device 102, it is understood thatmultiple different client devices (of different entities and/or users),each similar to client device 102, may be in remote communication withweb server 104.

Staging server 16 includes a processor 120. While referred to in thesingular, processor 120 may include any suitable number of processors ofone or more types (e.g., one or more central processing units (CPUs),etc.). Generally, processor 120 is configured to execute softwareinstructions stored in one or more memories (e.g., stored in apersistent memory such as a hard drive or solid state memory) of stagingserver 106. The software instructions, when executed by processor 120,implement a threading unit 122 and a neural network unit 124, which maycorrespond to threading unit 24 and neural network unit 26,respectively, of FIG. 1. In some embodiments, threading unit 122 and/orneural network unit 124 is/are part of a larger application or set ofapplications, which pre-processes electronic documents of all sorts forvarious purposes in addition to conversation threading. For example,such an application or application set may convert newly loadedelectronic documents to a pdf format, assign identifiers/labels to newlyloaded documents, implement textual and/or conceptual de-duplication ofdocuments, and so on.

A communication corpus 130 and a test corpus 132 may correspond tocommunication corpus 12 and test corpus 32, respectively, of FIG. 1.Each of communication corpus 40 and test corpus 132 may be stored in oneor more persistent memories. In some embodiments, communication corpus130 and/or test corpus 132 is/are stored in locations distributed acrossa large geographic area.

In a manner similar to that discussed above in connection with FIG. 1,electronic communication documents and other data in test corpus 132 maybe used by neural network unit 124 to train an artificial neuralnetwork. Thereafter, when neural network unit 124 processes documents ofcommunication corpus 130, the resulting data (e.g., data indicatingsegment locations, segment section locations, field locations, and/orfield values) may be passed to threading unit 122 to enable threadingunit 122 to arrange documents from communication corpus 130 intoconversation threads. Threading unit 122 may then generate metadataindicating the ordered relationship among documents within each thread.The metadata may be stored in communication corpus 130 in associationwith the appropriate documents, or in another suitable corpus ordatabase, for example.

Web server 104 includes a processor 140. As with processor 120,processor 140 may include any suitable number of processors and/orprocessor types. Generally, processor 140 is configured to executesoftware instructions stored in one or more memories (e.g., stored in apersistent memory such as a hard drive or solid state memory) of webserver 104.

Web server 104 includes a data storage 142 (e.g., one or more persistentmemories) that stores one or more web pages of an electronic documentreview (EDR) website 144. EDR website 144 includes instructions of theweb pages (e.g., HyperText Markup Language (HTML) instructions,JavaScript instructions, JavaServer Pages (JSP) instructions, and/or anyother type of instructions suitable for defining the content andpresentation of the web page(s)), and/or may include instructions of aplug-in, extension, and/or stand-alone software component that may bedownloaded by client device 102. EDR website 144, or another applicationor unit of web server 104 that is not shown in FIG. 2, may also includeinstructions for communicating with communication corpus 130 (andpossibly another corpus/database including metadata generated bythreading unit 122) as needed to obtain or modify the data storedtherein. In other embodiments, web server 104 accesses communicationcorpus 130 only indirectly, such as through staging server 106 (e.g., bysending requests for data to staging server 106) or another server.

Generally, EDR website 144 provides users accessing EDR website 144 witha browser-based user interface that enables the review of documents incommunication corpus 130. To this end, EDR website 144 may includeinstructions of a document display unit 146 that enables a user toreview the content of specific, selected documents via his or her webbrowser. EDR website 144 may also include instructions configured torecognize various inputs from users, and to act accordingly (e.g., todownload and/or display another document in response to the userselecting the document, and/or to save user tags/designations fordocuments to communication corpus 130, etc.). In some embodiments, EDRwebsite 144 also includes instructions of a format definition unit 148.Format definition unit 148 may provide a user interface via whichindividuals at remote client devices, such as client device 102, canprovide data defining/specifying particular header formats (e.g., headerformats for particular software clients and/or particular softwareversions). Format definition unit 148 is discussed in further detailbelow.

Client device 102 may be a laptop computer, a desktop computer, atablet, a smartphone, or any other suitable type of computing device. Inthe embodiment of FIG. 2, client device 102 includes a processor 150, arandom-access memory (RAM) 152, one or more input devices 154, a display156, a program storage 160, and a data storage 162. As with processors120 and 140, processor 150 may include any suitable number of processorsand/or processor types. Processor 150 may include one or more CPUs andone or more graphics processing units (GPUs), for example. Generally,processor 150 is configured to execute software instructions stored inprogram storage 160. Program storage 160 may include one or morepersistent memories (e.g., a hard drive and/or solid state memory), andstores a number of applications including a web browser application 164.Data storage 162 may also include one or more persistent memories, andgenerally stores data used by applications stored in program storage160. For example, data storage 162 may store local copies of electroniccommunication documents that were downloaded from communication corpus130 via web server 104.

Input device(s) 154 may include components that are integral to clientdevice 102, and/or exterior components that are communicatively coupledto client device 102, to enable client device 102 to accept inputs fromthe user. For example, input device(s) 154 may include a mouse, akeyboard, a trackball device, a microphone, etc. Display 156 may also beeither integral or external to client device 102, and may use anysuitable display technology (e.g., LED, OLED, LCD, etc.). In someembodiments, input device(s) 154 and display 156 are integrated, such asin a touchscreen display. Generally, input device(s) 154 and display 156combine to enable a user to interact with user interfaces provided byclient device 102.

RAM 152 stores portions of the instructions and data stored by programstorage 160 and data storage 162 when processor 150 executesapplications stored in program storage 160. When processor 150 executesweb browser application 164, for example, RAM 152 may temporarily storethe instructions and data required for its execution. In FIG. 2, webbrowser application 164 (while being executed) is represented in theprogram space of RAM 152 as web browser application 170. When the userof client device 102 uses web browser application 164 to access EDRwebsite 144, any scripts or other instructions of EDR website 144 (e.g.,instructions associated with document display unit 146, and possiblyformat definition unit 148) may be stored as a local copy in RAM 152.FIG. 2 illustrates a scenario where EDR website 144 is stored in RAM 152as EDR website 172, document display unit 146 is stored in RAM 152 asdocument display unit 174, and format definition unit 148 is stored inRAM 152 as format definition unit 176. Web browser application 170 mayinterpret the instructions of each of the local copies to present thepage(s) of EDR website 144 to the user, and to handle user interactionswith the page(s) as discussed further below. When various functions oractions are attributed herein to EDR website 172, document display unit174, or format definition unit 176, it is understood that those actionsmay be viewed as being caused by web server 104, by way of providing theinstructions of EDR website 144, document display unit 146, or formatdefinition unit 148, respectively, to client device 102 via network 110.

In operation, the user of client device 102, by operating inputdevice(s) 154 and viewing display 156, opens web browser application 164to access EDR website 144 for purposes of reviewing (and possiblydesignating categories or classifications of) electronic documents. Tofully access EDR website 144, the user may be required to satisfycertain security measures, such as entering a valid login and password,for example. The user may then utilize a web page of EDR website 144 toindicate the project or workspace that he or she wishes to access. Webserver 104 may use the indication of the project or workspace toidentify the appropriate set of documents in communication corpus 130,or to identify the entirety of communication corpus 130 (e.g., if corpus130 only includes electronic communication documents for a singleproject or workspace).

By the time the user of client device 102 accesses EDR website 144, thedocuments in communication corpus 130 may already have beenpre-processed by staging server 106. For example, threading unit 122 ofstaging server 106 may have previously identified which electroniccommunication documents belong to which threads and thread groups, andmay have stored metadata indicative of those relationships (e.g.,fingerprints) in communication corpus 130 or another database.

In an embodiment, when the user of client device 102 selects a specificelectronic communication document (e.g., from a list of documentidentifiers presented by EDR website 172, and each corresponding to adocument in communication corpus 130), web server 104 retrieves theelectronic communication document from communication corpus 130, alongwith associated metadata indicating thread-related information. Webserver 104 may then transmit the document and metadata to client device102, where document display unit 174 may cause the text (and possiblyimages) of the selected electronic communication document to bepresented to the user via a graphical user interface (GUI) on display156. EDR website 172 may also cause thread-related information to bepresented to the user on display 156. For example, web server 104 orclient device 102 may use the thread-related metadata to present to theuser a display indicative of the ordered relationship among documents inone or more threads (e.g., an indented list of document identifiers withthe first level of indentation corresponding to a root document of athread, and/or a visualization that graphically depicts the relationshipamong documents within a thread, etc.).

In some embodiments, a user can code the electronic communicationdocuments that he or she is reviewing according to certain predefinedand/or user-created tags/designations, such as “privilege,” “noprivilege,” “responsive,” “not responsive,” and so on. In someembodiments, user changes to the designations for an electroniccommunication document are communicated to web server 104, whichmodifies the document designation appropriately (e.g., withincommunication corpus 130 or another location, depending upon where suchdata is stored). Web server 104 may directly modify the designation, ormay request that another device or system (e.g., staging server 106) doso.

In some embodiments, the user of client device 102 (i.e., the reviewinguser), or a user of another, similar client device remote from webserver 104 (e.g., a client device of an administrator employed by thesame entity/customer as one or more reviewing users), may provideinformation that facilitates the accurate pre-processing of electroniccommunication documents. Specifically, the user may provide informationthat staging server 106 can utilize to better tailor the neural network,and/or header parsing operations, to the collection of documents that aparticular customer, user, or set of users expects to encounter.

To obtain this information, format definition unit 148 may generate oneor more interactive controls, within EDR website 144, that enable a userto upload/submit one or more sets of header definition data to webserver 104. Alternatively, format definition unit 148 may be included ina website hosted by staging server 106 (or another server not shown inFIG. 2), or may be included in software that is installed at clientdevice 102 and configured to send information directly or indirectly tostaging server 106. Each set of header definition data may define one ormore characteristics of a respective header format (e.g., for headersgenerated by a particular software client and/or version). For example,the header definition data may specify one or more header field keywordsand/or header field positions.

The header definition data may take various different forms, dependingon the embodiment. For instance, the user of client device 102, oranother client device, may enter a set of rules, and/or a set of regularexpressions (e.g., specifying partial keywords and wildcards), that maybe used to parse headers. The user of client device 102 may thenactivate one or more of the controls generated by format definition unit148 (or more precisely, by the local format definition unit 176) tosubmit the header definition data to web server 104, which may in turnforward the header definition data to staging server 106. Staging server106 may then parse headers using the header definition data to identifyparticular header fields and their values (e.g., for use by threadingunit 122 to generate conversation threads).

In an alternative embodiment, format definition unit 148 (or a similarunit of a website hosted by staging server 106, or another server notshown in FIG. 2) may provide a graphical user interface (GUI) thatenables individuals (e.g., reviewing users or administrators associatedwith customers, or system developers or administrators associated withweb server 104 and/or staging server 106) to assist the operations ofstaging server 106. The GUI may provide an easy and convenient mechanismwhereby an individual's selection of particular header fields areautomatically translated into format definition data. Alternatively, theGUI may provide a mechanism whereby an individuals' selection ofparticular segments and/or segment portions are automatically translatedinto label data associated with documents that are then added to testcorpus 132 and used to train the neural network. One example of such aGUI is discussed below in connection with FIG. 7.

While FIG. 2 shows an embodiment in which an electronic document reviewtool is provided as a web-based service, it is understood that otherembodiments are also possible. For example, program storage 160 ofclient device 102 may store a software product that enables clientdevice 102 to interface directly with staging server 106, withoutrequiring web server 104, or to interface with another server (not shownin FIG. 2) that acts as an intermediary between staging server 106 andany client devices. In still another embodiment, a software productinstalled at client device 102 may enable client device 102 to directlyimplement the functions of staging server 106.

Moreover, the various components of the environment 100 may interoperatein a manner that is different than that described above, and/or theenvironment 100 may include additional components not shown in FIG. 2.For example, an additional platform/server may act as an interfacebetween web server 104 and staging server 106, and may perform variousoperations associated with providing the threading and/or other servicesof staging server 106 to web server 104 and/or other web servers.

III. Example Artificial Neural Network

FIG. 3 depicts an example artificial neural network 200 that may betrained by neural network unit 26 of FIG. 1 or neural network unit 124of FIG. 2, according to one embodiment and scenario. The example neuralnetwork 200 includes neurons arranged in multiple layers, including aninput layer 202, one or more hidden layers 204-1 through 204-M, and anoutput layer 206. Each of the layers in neural network 200 may have anydesired number of layers (e.g., j and k in FIG. 3 may be any positiveintegers). It is understood that the present invention may use neuralnetworks that have different configurations and/or connections thatthose shown in FIG. 3, before and/or after training of the neuralnetwork.

Input layer 202 may correspond to various different characteristics oftext in a given electronic communication document. For example, inputlayer 202 may include a first input (x₁) representing a total number oflines of text in the document, a second input (x₂) representing aposition of a first line break or carriage return in the text (or avector or array representing locations of all line breaks in the text,etc.), a third input (x₃) representing a position of a first colon inthe text (or a vector or array representing locations of all colons inthe text, etc.), and so on. Input layer 202 may include tens, hundreds,or even thousands of inputs, for example. In some embodiments, however,the number of inputs actually used by neural network 200 decreases afterthe training process, as discussed further below.

Each of the neurons in the hidden layer(s) 204-1 through 204-M mayoperate on one of more of the inputs from input layer 202, and/or one ormore outputs from a previous one of the hidden layers, to generate adecision or other output. Output layer 206 may include one or moreoutputs each indicating the location of a particular segment or segmentportion within the document being processed. In some embodiments,however, outputs of neural network 200 may be obtained not just fromoutput layer 206, but also from one or more of hidden layer(s) 204-1through 204-M. For example, each successive layer may examine thedocument under consideration at a finer level of detail. In one suchembodiment where M=3, for example, hidden layer 204-1 may examinevarious inputs to determine delineations between multiple conversationsegments in a single document, hidden layer 204-2 may examine variousoutputs of hidden layer 204-1 (and possibly also one or more inputs ofthe input layer) to determine locations of a header, message body,and/or signature block within each identified segment, and hidden layer204-3 may examine specific segment sections identified by hidden layer204-2 to determine locations of particular fields within those sections.In other embodiments, the functions of each layer are not as neatlydelineated. For example, two or more of hidden layer(s) 204-1 through204-M may make decisions relating to segment locations, with one ofthose layers also making decisions relating to specific field locations,and so on.

In some embodiments, neural network 200 is a recurrent neural network,with decisions or outputs from one or more layers of neural network 200being fed back to one or more previous layers (e.g., the immediatelypreceding layer) during training, in order to provide an indication ofthe importance of various parameters to a particular decision orcalculation. For example, training unit 30 of FIG. 1 may set weights forparticular inputs of input layer 202, and/or for outputs of particularneurons in one or more of hidden layer(s) 204-1 through 204-M, based onsuch feedback. In addition, or alternatively, feedback of this sort maybe used to identify neurons that are irrelevant, or of insignificantrelevance, to the determination of the desired outputs of neural network200. Once the training process is complete, in some embodiments, thoseneurons may be bypassed in order to reduce the amount of processingresources and/or processing time required for each document.

FIG. 4 depicts an example neuron 220 that may correspond to the neuronlabeled as “1,1” in hidden layer 204-1 of FIG. 3, according to oneembodiment and scenario. Each of the inputs to neuron 220 (in thisexample, each of the inputs within input layer 202 of FIG. 3) may beweighted according to a set of weights (w₁ through w_(i)) determinedduring the training process (e.g., if neural network 200 is a recurrentneural network), and then applied to a summing node 222 of neuron 220.While FIG. 4 shows all inputs x₁ through x_(i) being associated with aweight, in some scenarios and/or embodiments weights are not determinedfor certain inputs. Moreover, certain inputs that were identified asinsignificant to the accurate determination of outputs (or as beingbelow a threshold level of significance) may be ignored by neuron 220.

The sum of the weighted inputs, z₁, may be input to a function 224,labeled in FIG. 4 as F_(1,1)(z₁). The function 224 may represent anysuitable linear or non-linear operation on z₁. As shown in FIG. 4, theoutput of function 224 may be provided to a number of neurons of thenext layer, and/or may be provided as an output of neural network 200.For example, the output may indicate a location of a segment or segmentportion, or may be a parameter that is calculated or determined as aninterim step when determining such a location.

In other embodiments, and/or in other training scenarios, neuron 220 maybe arranged differently than is shown in FIG. 4. For example, summingnode 222 may be omitted, and function 224 may operate directly on one ormore of the inputs x₁ through x_(i). As another example, neuron 220 maynot apply weights to any of the inputs x₁ through x_(i).

IV. Example Processing of an Electronic Communication

DOCUMENT

The specific manner in which the neural network employs machine visionto identify particular segments and/or segment portions may, of course,vary depending on the content and labeling of training documents withinthe test corpus (e.g., test corpus 32 of FIG. 1 or test corpus 132 ofFIG. 2), as well as the set of document characteristics that are chosen(e.g., by a system developer and/or a customer) to serve as inputs tothe neural network for purposes of training. The initial structure ofthe neural network (e.g., the type of neural network, the number oflayers, the number of neurons per layer, etc.) typically also affectsthe manner in which the trained neural network processes a document.Moreover, the manner in which the trained neural network processes adocument can be very complex, and/or non-intuitive. For the sake ofillustration, however, some relatively simple, intuitive examples of howa trained neural network may use machine vision to process a documentwill now be discussed, in connection with FIG. 5.

FIG. 5 depicts text-based content of an example electronic communicationdocument 250, which may be processed using an artificial neural network,such as neural network 200 of FIG. 3 or a different neural networkgenerated by neural network unit 26 of FIG. 1 or neural network unit 124of FIG. 2, for example. While electronic communication document 250 isshown in the form that it might appear to a reviewing user, the term“text-based content,” as used herein, refers not only to thealphanumeric characters of electronic communication document 250, butalso to any formatting or control elements used to generate thepresentation shown in FIG. 5 (e.g., line spacing, line breaks, characterfonts, etc.). If electronic communication document 250 represents anemail, for example, the text-based content may include HTML headingelements (e.g., “<h>”), paragraph elements (e.g., “<p>”), line breakelements (e.g., “<br>”), and so on. As another example, the text-basedcontent may include ASCII printable characters as well as ASCII controlcharacters, such as horizontal tab characters (hexadecimal value 09),carriage return characters (hexadecimal value 13), and so on.

As seen in FIG. 5, electronic communication document 250 includes fourconversation segments 252A through 252D, which include respectiveheaders 254A through 254D, message bodies 256A through 256D, andsignature blocks 258A through 258D. As noted above, in some embodiments,layers of the neural network may successively examine a document atincreasing levels of granularity. Thus, for example, a first layer orset of layers in the neural network may identify the locations ofsegments 252A through 252D. The neurons of the layer(s) may outputcharacter numbers, line numbers, and/or other indicators of the startand/or end of each of segments 252A through 252D, or may add adelineation tag to the corresponding locations within a copy ofelectronic communication document 250, for example. Similarly, a secondlayer or set of layers may identify the locations of headers 254Athrough 254D, and a third layer or set of layers may identify thelocations of particular fields within those headers (e.g., the date/timeof sending, the sender, and/or the recipient(s)). In other embodiments,locations of the message bodies 256A through 256D, the signature blocks258A through 258D, and/or fields or portions thereof, may also, orinstead, be identified by layers of the neural network. Further,different embodiments may define various sections in different ways. Forexample, the name “Elgar” in signature block 258C may instead be viewedas a part of message body 256C.

To identify the locations of segments 252A through 252D, the trainedneural network might, for example, identify all lines that includeexactly one colon and, for each such line, consider (1) the number ofcontiguous, immediately preceding lines that do not include a colon, (2)the number of words following the colon in the same line, (3) whethereach of the words (or at least two of the words, etc.) following thecolon, and in the same line as the colon, has the first lettercapitalized, and so on. For instance, lines that include exactly onecolon, are immediately preceded by at least three lines with no colons,and have exactly two to four words after the colon in the same line(with at least two of the words having only the first lettercapitalized) may each be viewed as the first line of a new segment.

The neural network might also follow parallel paths for identifyingsegments, particularly if the neural network was trained using documentsgenerated by different software clients (and/or different versions of asoftware client, and/or documents with field names in differentlanguages). For example, the neural network might also identify allinstances of at least two contiguous blank lines, and examine variouscharacteristics of the text immediately preceding and/or following thoseblank lines.

To identify the locations of headers 254A through 254D within segments252A through 252D, the trained neural network might assume that eachsegment begins with a header, and identify the end of each header usingvarious pieces of information within the text-based content. Forexample, the neural network may identify the end of the header as thefirst blank line, within a given segment, occurring after the first lineof that segment.

To identify the locations of particular fields within headers 254Athrough 254D, the trained neural network might identify instances inwhich one or two words at the beginning of a line in a given header areimmediately followed by a colon, and then search for particular keywords(e.g., “From,” “Sender,” or “Author” for a sender of the messagecorresponding to that segment) within the word or words preceding thecolon in those lines. In some embodiments, the neural network has accessto a library of potential keywords, which may or may not have beengenerated or modified during the training process, depending on theembodiment. In some embodiments and scenarios, different libraries areavailable, with each library including keywords in a different language(e.g., English, Spanish, French, German, Russian, etc.). In someembodiments and/or scenarios, the neural network uses a priority oflanguages to attempt to identify keywords in a particular order (e.g.,first using English, then Spanish if that is unsuccessful, etc.),thereby saving processing resources in instances where certain languageare more likely to be encountered than others. The language priority maybe indicated by a system developer or by a customer via a user interface(e.g., as discussed further below), or may be automatically determinedduring training of the neural network, for example.

In some embodiments, the neural network also, or instead, generatesindicators of the likelihood or confidence that the location of aparticular segment or segment portion is correct. For example, some orall of the location indicators generated for electronic communicationdocument 250 may be associated with a confidence score between 1 and100. In some embodiments, the confidence scores may be used by athreading unit (e.g., threading unit 24 of FIG. 1 or threading unit 122of FIG. 2) to calculate a likelihood or confidence that electroniccommunication document 250 belongs to a particular thread. Electroniccommunication document 250 may be added to the thread only if thelikelihood or confidence exceeds some threshold level, for example.

Alternatively, or in addition, the likelihood or confidence level thatelectronic communication document 250 belongs to a particular thread,and/or is at a particular position within the thread, may be displayedto a reviewing user (e.g., via EDR website 172 of FIG. 2). This may helpto avoid user confusion in certain situations, such as when a userencounters an electronic communication document with a message body thatdoes not make sense in the context of a particular thread. Further, thisapproach may avoid the common misconception among users that threadingis an exact science, or that thread information presented to a user isnecessarily complete and error-free.

In some embodiments, the threading unit can consider whether to addelectronic communication document 250 to a particular thread even if theneural network has not successfully generated all of its outputs. Forexample, the neural network may successfully identify segments 252Athrough 252D, as well as the sender, recipient, and date/time fields ofheaders 254A, 254B, and 254D, but fail to identify all of thecorresponding fields in header 254C due to the different format of thatheader. Nonetheless, the threading unit may be able use the incompleteoutput of the neural network to add electronic communication document250 to a particular thread. This may provide an important advantage overconventional techniques, which typically discard or ignore documentsthat cannot be fully parsed.

V. Example Method for Identifying Portions of Electronic CommunicationDocuments

FIG. 6 is a flow diagram of an example method 280 for identifyingportions of electronic communication documents. The method 280 may beimplemented by one or more processors of a computing device or system,such as processor 120 of staging server 106 in FIG. 2, for example. Inthe method 280, an artificial neural network is trained (block 282) toidentify conversation segments, and/or portions of conversationsegments, within electronic communication documents (e.g., emails). Thesegment “portions” may include specific types of segment sections (e.g.,header, message body, and/or signature block), and/or may includespecific types of fields within segments (e.g., specific header fields).The neural network may be trained by analyzing a test set of electroniccommunication documents that have each previously been labeled or taggedto indicate positions of segments and/or segment portions. For example,block 282 may include comparing one or more interim position indicatorsthat were generated by the neural network (when analyzing a firstdocument in the test set) to one or more known position indicatorscorresponding to that first document.

The neural network (e.g., a neural network similar to neural network 200of FIG. 3) may include multiple layers, including an input layer with anumber of different input parameters that each correspond to a differentcharacteristic of text-based content. The “text-based content” mayinclude both alphanumeric characters that can be viewed by a personreading a given document, and formatting or control elements thatspecify the presentation (e.g., line spacing, line breaks, font, etc.)of the alphanumeric characters. The characteristics that correspond tothe input parameters may be of any sort that could potentially be usefulfor discerning the location of segments and/or particular segmentportions. For example, there may be hundreds of characteristics, such ascharacteristics indicative of line break positioning, characteristicsindicative of line spacing, characteristics indicative of charactertypes (e.g., font, font size, etc.), characteristics indicative ofcharacter counts (e.g., per line), characteristics indicative of colonpositioning, and/or any number of other suitable characteristics.

The neural network may be a recurrent neural network. Moreover, thetraining performed at block 282 may include determining weights that theneural network will apply to one or more of the input parameters, and/orto one or more outputs generated by neurons of one or more of the neuralnetwork layers.

An electronic communication document (e.g., another email) that includestext-based content is then received (block 284). The electroniccommunication document may or may not be of the same type (e.g.,generated by the same software client and version) as one or more of thetraining documents used at block 282. The electronic communicationdocument may be retrieved from a local or remote database (e.g.,communication corpus 12 of FIG. 1 or communication corpus 130 of FIG.2), or may be pushed to a server implementing the method 280 by anotherserver, for example.

The text-based content of the document received at block 284 is thenprocessed (block 286) using the trained neural network. The processingmay include generating, within the layers of the neural network, one ormore position indicators for the electronic communication document. Inparticular, the processing may include generating one or more segmentindicators denoting positions of one or more conversation segmentswithin the electronic communication document (block 288), and/orgenerating one or more segment portion indicators denoting positions ofone or more portions of one or more conversation segments within theelectronic communication document (block 290). Segment “portion”indicators may include indicators of the locations of particular segmentsections (e.g., headers, and/or message bodies, etc.), and/or indicatorsof the locations of particular fields (e.g., particular header fields).

In some embodiments, successive layers of the neural network process thedocument at increased levels of granularity. For example, a first layerof the neural network may generate one or more segment indicatorsdenoting positions of different segments, a second layer of the neuralnetwork may generate one or more segment section indicators denotingpositions of different segment sections (e.g., headers, message bodies,and/or signature blocks), and a third layer of the neural network maygenerate one or more field indicators denoting positions of differentfields (e.g., different header fields).

An ordered relationship between the electronic communication documentand one or more other electronic communication documents is determined(block 292) using the position indicators (i.e., the segment and/orsegment portion position indicators) that were generated at block 286.The ordered relationship may be determined as a part of a conversationthreading process implemented by a threading unit (e.g., threading unit24 of FIG. 1 or threading unit 122 of FIG. 2). For example, the positionindicator(s) generated at block 286 may include indicators for thepositions of particular header fields (e.g., sender, recipient, and/ordate/time) within each of one or more conversation segments of thedocument, and block 292 may include determining values of those fieldsas well as hashing those values. Block 292 may also include generatingand storing metadata indicative of the ordered relationship.

VI. Example Graphical User Interface for Facilitating Identification ofPortions of Electronic Communication Documents

As noted above in connection with FIG. 2, a GUI may be provided toenable individuals to assist the operations of staging server 106. Onesuch GUI 300, corresponding to one embodiment and scenario, is depictedin FIG. 7. In one embodiment, GUI 300 is presented on display 156 ofFIG. 2 via web browser application 170 and format definition unit 176 ofEDR website 172. In other embodiments, GUI 300 is provided by adifferent website (e.g., a website that can only be accessed by anadministrator of a customer, etc.) or software application. Moreover, insome embodiments, GUI 300 may also, or instead, by presented on adisplay of a client device associated with an entity that owns and/ormaintains staging server 106 (e.g., to enable a software developer,technical sales representative, etc., to perform the user operationsdescribed below). For ease of explanation, however, the description ofFIG. 7 that follows assumes that GUI 300 is generated by EDR website172.

In the example GUI 300, a document review pane 302 shows the text of anelectronic communication document 304 (e.g., an email). Document reviewpane 302 may be generated by document display unit 174 of FIG. 2, forexample, and format definition unit 176 may communicate with documentdisplay unit 174 to enable some or all of the user interactionsdescribed below. Alternatively, document review pane 302 may begenerated directly by format definition unit 176.

GUI 300 also includes a user prompt pane 306 that guides a user througha process for tagging/labeling specific conversation segments and/orsegment portions within electronic communication document 304. Inparticular, user prompt pane 306 presents to the user a prompt 320indicating which action the user should take next with respect toelectronic communication document 304. The prompt 320 may generallyinstruct the user to select or tag a particular portion of electroniccommunication document 304. The user may select or tag that portion,within document review pane 302, by left-clicking and dragging a mouse,for example, or using touch inputs or any other suitable method ofselection.

After selecting a portion of electronic communication document 304 asinstructed, the user may select an interactive control 322 within userprompt pane 306 to confirm the selection. A new prompt similar to prompt320 may then appear or, if all relevant sections have been tagged, theprompting process may end. In alternative embodiments, interactivecontrol 322 is in another location (e.g., in document review pane 302,or in a right-click menu item, etc.), or is not included in GUI 300 atall (e.g., if the selection requires no user confirmation).

In the example embodiment of FIG. 7, the user is prompted to selectconversation segments (in chronological order from newest to oldest)and, for each segment, to select the header, the author/sender/fromfield, the recipient/to field, the data field, and the time field. Inother embodiments, other segment sections and/or fields may be selected,older segments may be tagged before newer segments, the date and timemay instead be considered a single field, and so on.

In some embodiments, document review pane 304 displays an indicator ofthe current user selection, and/or of some or all of the previous userselections. In the example GUI 300, for instance, indicatorscorresponding to selections of and within a current conversation segmentpersist until a new conversation segment is chosen. Thus, it is seen inFIG. 7 that a past user selection of a second conversation segment(corresponding to the seventh prompt) is indicated by an indicator 324,and a current user selection of a header within that segment(corresponding to the eight, current prompt) is indicated by anindicator 326. In other embodiments, indicators may have a differentappearance, may appear at different times, or may be omitted entirely(e.g., other than highlighting or some other brief indicator during theselection operation, to let the user know what he or she has justselected).

In the embodiment of FIG. 7, each prompt persists in user prompt pane306 even after the user has taken the corresponding action. In otherembodiments, a prompt does not persist after the user has selected aportion of electronic communication document 304 and activatedinteractive control 322. Moreover, in some embodiments, user prompt pane306 may be omitted and prompt 320 may instead be shown in documentreview pane 302.

In embodiments that utilize neural networks (e.g., the embodiment ofFIG. 1 and/or 2), the user selections made in response to the series ofprompts may trigger the automatic generation of metadata that labels ortags the corresponding portions of electronic communication document304. For example, format definition unit 176 of FIG. 2 may detect theuser selections, and in response automatically generate metadataspecifying which portions of electronic communication document 304correspond to which segments/headers/fields. Formation definition unit176 may then transmit the metadata, along with electronic communicationdocument 304, to web server 104, which may in turn store (or causestaging server 106 to store) the metadata and electronic communicationdocument 304 in test corpus 132 for use in training the neural network(e.g., for use during the initial training of the neural network, or ina subsequent training/refining stage). In some embodiments andscenarios, electronic communication documents and metadata received viaformat definition unit 176 are more heavily weighted during thetraining, under the assumption that those documents are mostrepresentative of the specific set of documents that will beprocessed/threaded.

In some embodiments that parse headers (instead of, or in addition to,using a neural network), the user selections made in response to theseries of prompts cause format definition unit 176 to automaticallygenerate a set of parsing rules. The rules may be generated based on thekeywords of fields selected by the user, for example, and/or based onthe relative positions of selected header fields, for example. Formationdefinition unit 176 may then transmit the rules to web server 104, whichmay in turn forward the rules to staging server 106 for use in headerparsing (e.g., for purposes of threading electronic conversationdocuments in communication corpus 130).

In some embodiments, GUI 300 (or another interface or mechanism) alsoenables a user to indicate a priority of languages. For example, acustomer may know that most documents for a particular project will bein English, that a large minority of the documents will be in French,and a smaller minority of the documents will be in Russian, German, orSpanish. The user may indicate the expected order using one or moreinteractive controls (e.g., a series of boxes for typing in languages,or a menu enabling the ordered selection of multiple languages, etc.).The controls may be provided to the user within user prompt pane 306,for example.

Format definition unit 176 may detect the user-indicated order oflanguages, and transmit the order to web server 104, and web server 104may forward the indicated order to staging server 106. Staging server106 may then parse headers for keywords according to the differentlanguages, starting with the most likely/common language and proceedinguntil an expected keyword or set of keywords is found (e.g., one keywordper conversation segment identified in a given document, etc.). Byavoiding the need to parse the headers of each document according to allpossible languages, less time (and/or fewer processing resources instaging server 106) may be required for the threading process.

VII. Example Method for Facilitating Recognition of Header Fields inElectronic Communication Documents

FIG. 8 is a flow diagram of an example method 400 for facilitating therecognition of header fields in electronic communication documents(e.g., for a threading process). The method 400 may be implemented byone or more processors of a computing device or system, such asprocessor 120 of staging server 106 of FIG. 2, for example. In themethod 400, header definition data is received (block 410) from a remoteclient device. The header definition data defines one or morecharacteristics of a particular header format. For example, the headerdefinition data may define one or more header field keywords (e.g.,header field labels/names) and/or partial keywords (e.g., partial headerfield labels/names with wildcards), and/or positions of one or moreheader field keywords. The header field or fields may include a senderfield (e.g., “Author,” or “From,” etc.), a recipient field (e.g.,“Recipient,” or “To,” etc.), a date field, and/or a time field (e.g., a“Sent” field indicating both date and time), for example.

In some embodiments, the header definition data is received directlyfrom the remote client device (e.g., from client device 102 via network110 of FIG. 2). Alternatively, the header definition data may bereceived via one or more other computing devices or systems, such as webserver 104 of FIG. 2. The header definition data may include a set ofone or more rules, and/or may include one or more regular expressions(e.g., including one or more partial keywords and wildcards), thatdefine the characteristic(s) of the header format.

An electronic communication document (e.g., an email) that includestext-based content is received (block 420). The electronic communicationdocument may be retrieved from a local or remote database (e.g.,communication corpus 12 of FIG. 1 or communication corpus 130 of FIG.2), or may be pushed to a server implementing the method 400 by anotherserver, for example. The “text-based content” may include bothalphanumeric characters that can be viewed by a person reading a givendocument, and formatting or control elements that specify thepresentation (e.g., line spacing, line breaks, font, etc.) of thealphanumeric characters.

Using the header definition data received at block 410, values of theone or more header fields (for which keywords and/or partial keywordsare defined by the header definition data) within the text-based contentare determined (block 430). If the header definition data includes wholeor partial keywords, for example, those keywords or partial keywords maybe used to search for the appropriate field values. As another example,if the header definition data includes language priority data, theheader keywords for each language may be used, in priority order, one ata time until an expected number of keywords are found (e.g., one persegment, or three per segment, etc.).

Using the header field value(s) determined at block 430, an orderedrelationship between the electronic communication document (received atblock 420) and one or more other electronic communication documents isdetermined (block 440). The ordered relationship may be determined as apart of a conversation threading process implemented by a threading unit(e.g., threading unit 24 of FIG. 1 or threading unit 122 of FIG. 2). Forexample, block 440 may include hashing the header field value(s)determined at block 430.

Metadata indicating the ordered relationship, which may be generatedduring block 440, is stored in a memory (block 450). For example, themetadata may be stored in communication corpus 130 of FIG. 1, inassociation with a copy of the electronic communication document thatwas received at block 420.

In some embodiments, the method 400 may also include one or moreadditional blocks, and/or certain blocks may be omitted. For example,the method 400 may include an additional block in which a user interface(e.g., GUI 300 of FIG. 7) is caused to be provided to a user at theremote client device. In such an embodiment, the header definition datareceived at block 410 may include data that was generated based on userinputs entered via the user interface. As another example, block 450 maybe omitted from the method 400. As yet another example, the method 400may include an additional block in which an electronic document reviewtool is caused to present (e.g., to a user of the remote client device)an indication of the ordered relationship determined at block 440.

VIII. Additional Considerations

The following additional considerations apply to the foregoingdiscussion. Throughout this specification, plural instances mayimplement operations or structures described as a single instance.Although individual operations of one or more methods are illustratedand described as separate operations, one or more of the individualoperations may be performed concurrently, and nothing requires that theoperations be performed in the order illustrated. These and othervariations, modifications, additions, and improvements fall within thescope of the subject matter herein.

Unless specifically stated otherwise, discussions herein using wordssuch as “processing,” “computing,” “calculating,” “determining,”“presenting,” “displaying,” or the like may refer to actions orprocesses of a machine (e.g., a computer) that manipulates or transformsdata represented as physical (e.g., electronic, magnetic, or optical)quantities within one or more memories (e.g., volatile memory,non-volatile memory, or a combination thereof), registers, or othermachine components that receive, store, transmit, or displayinformation.

As used herein any reference to “one embodiment” or “an embodiment”means that a particular element, feature, structure, or characteristicdescribed in connection with the embodiment is included in at least oneembodiment. The appearances of the phrase “in one embodiment” in variousplaces in the specification are not necessarily all referring to thesame embodiment.

As used herein, the terms “comprises,” “comprising,” “includes,”“including,” “has,” “having” or any other variation thereof, areintended to cover a non-exclusive inclusion. For example, a process,method, article, or apparatus that comprises a list of elements is notnecessarily limited to only those elements but may include otherelements not expressly listed or inherent to such process, method,article, or apparatus. Further, unless expressly stated to the contrary,“or” refers to an inclusive or and not to an exclusive or. For example,a condition A or B is satisfied by any one of the following: A is true(or present) and B is false (or not present), A is false (or notpresent) and B is true (or present), and both A and B are true (orpresent).

In addition, use of “a” or “an” is employed to describe elements andcomponents of the embodiments herein. This is done merely forconvenience and to give a general sense of the invention. Thisdescription should be read to include one or at least one and thesingular also includes the plural unless it is obvious that it is meantotherwise.

Upon reading this disclosure, those of skill in the art will appreciatestill additional alternative structural and functional designs foridentifying particular portions of electronic communication documentsthrough the principles disclosed herein. Thus, while particularembodiments and applications have been illustrated and described, it isto be understood that the disclosed embodiments are not limited to theprecise construction and components disclosed herein. Variousmodifications, changes and variations, which will be apparent to thoseskilled in the art, may be made in the arrangement, operation anddetails of the method and apparatus disclosed herein without departingfrom the spirit and scope defined in the appended claims.

The patent claims at the end of this patent application are not intendedto be construed under 35 U.S.C. § 112(f) unless traditionalmeans-plus-function language is expressly recited, such as “means for”or “step for” language being explicitly recited in the claim(s).

What is claimed:
 1. A computer-implemented method for identifyingportions of electronic communication documents, the computer-implementedmethod comprising: training, by one or more processors analyzing a testset of electronic communication documents, an artificial neural networkto identify portions of conversation segments, within electroniccommunication documents, wherein the artificial neural network includesa plurality of layers, and wherein an input layer of the plurality oflayers includes a plurality of input parameters each corresponding to adifferent characteristic of text-based content; receiving, by the one ormore processors, a first electronic communication document that includesfirst text-based content; processing, by the one or more processors, thefirst text-based content of the first electronic communication documentusing the trained artificial neural network, wherein processing thefirst text-based content includes generating, within the plurality oflayers, one or more position indicators for the first electroniccommunication document, and wherein the one or more position indicatorsinclude one or more segment portion indicators denoting positions of oneor more portions of one or more conversation segments within the firstelectronic communication document; and determining, by the one or moreprocessors, an ordered relationship between the first electroniccommunication document and one or more other electronic communicationdocuments using the one or more position indicators for the firstelectronic communication document.
 2. The computer-implemented method ofclaim 1, wherein training the artificial neural network includescomparing (i) one or more interim position indicators generated byanalyzing a first test document within the test set of electroniccommunication documents to (ii) one or more known position indicatorscorresponding to the first test document.
 3. The computer-implementedmethod of claim 1, wherein the plurality of input parameters includesparameters indicative of one or more of (i) line break positioning, (ii)line spacing, (iii) character types, or (iv) character counts.
 4. Thecomputer-implemented method of claim 1, wherein the artificial neuralnetwork is a recurrent neural network, and wherein training theartificial neural network includes determining weights that theartificial neural network will apply to one or both of (i) one or moreof the plurality of input parameters, and (ii) one or more outputsgenerated by one or more of the plurality of layers.
 5. Thecomputer-implemented method of claim 1, wherein: training the artificialneural network further includes training the artificial neural networkto identify one or more of (i) headers, (ii) message bodies, or (iii)signature blocks, within electronic communication documents; and the oneor more segment portion indicators include one or more segment sectionindicators denoting positions of one or more of (i) one or more headerswithin the first electronic communication document, (ii) one or moremessage bodies within the first electronic communication document, or(iii) one or more signature blocks within the first electroniccommunication document, and one or more field indicators denotingpositions of fields, within the first electronic communication document,that correspond to one or more field types.
 6. The computer-implementedmethod of claim 5, wherein the one or more position indicators furtherinclude one or more segment indicators denoting positions of one or moreconversation segments within the first electronic communicationdocument.
 7. The computer-implemented method of claim 6, whereinprocessing the first text-based content of the first electroniccommunication document using the trained artificial neural networkincludes: generating, by a first layer of the plurality of layers, theone or more segment indicators; generating, by a second layer of theplurality of layers, the one or segment section indicators; andgenerating, by a third layer of the plurality of layers, the one or morefield indicators.
 8. A computing system comprising: an electronicdocument database; one or more processors; and one or more memoriesstoring instructions that, when executed by the one or more processors,cause the computing system to train, by analyzing a test set ofelectronic communication documents, an artificial neural network toidentify portions of conversation segments, within electroniccommunication documents, wherein the artificial neural network includesa plurality of layers, and wherein an input layer of the plurality oflayers includes a plurality of input parameters each corresponding to adifferent characteristic of text-based content, retrieve, from theelectronic document database, a first electronic communication documentthat includes first text-based content, process the first text-basedcontent of the first electronic communication document using the trainedartificial neural network, wherein processing the first text-basedcontent includes generating, within the plurality of layers, one or moreposition indicators for the first electronic communication document, andwherein the one or more position indicators include one or more segmentportion indicators denoting positions of one or more portions of one ormore conversation segments within the first electronic communicationdocument, and determine an ordered relationship between the firstelectronic communication document and one or more other electroniccommunication documents using the one or more position indicators forthe first electronic communication document.
 9. The computing system ofclaim 8, wherein the instructions cause the computing system to trainthe artificial neural network at least by comparing (i) one or moreinterim position indicators generated by analyzing a first test documentwithin the test set of electronic communication documents to (ii) one ormore known position indicators corresponding to the first test document.10. The computing system of claim 8, wherein the plurality of inputparameters includes parameters indicative of one or more of (i) linebreak positioning, (ii) line spacing, (iii) character types, or (iv)character counts.
 11. The computing system of claim 8, wherein theartificial neural network is a recurrent neural network, and wherein theinstructions cause the computing system to train the artificial neuralnetwork at least by: determining weights that the artificial neuralnetwork will apply to one or both of (i) one or more of the plurality ofinput parameters, and (ii) one or more outputs generated by one or moreof the plurality of layers.
 12. The computing system of claim 8,wherein: the instructions cause the computing system to train theartificial neural network further by training the artificial neuralnetwork to identify one or more of (i) headers, (ii) message bodies, or(iii) signature blocks, within electronic communication documents; andthe one or more segment portion indicators include one or more segmentsection indicators denoting positions of one or more of (i) one or moreheaders within the first electronic communication document, (ii) one ormore message bodies within the first electronic communication document,or (iii) one or more signature blocks within the first electroniccommunication document, and one or more field indicators denotingpositions of fields, within the first electronic communication document,that correspond to one or more field types.
 13. The computing system ofclaim 12, wherein the one or more position indicators further includeone or more segment indicators denoting positions of one or moreconversation segments within the first electronic communicationdocument.
 14. The computing system of claim 13, wherein the instructionscause the computing system to process the first text-based content ofthe first electronic communication document using the trained artificialneural network at least by: generating, by a first layer of theplurality of layers, the one or more segment indicators; generating, bya second layer of the plurality of layers, the one or segment sectionindicators; and generating, by a third layer of the plurality of layers,the one or more field indicators.
 15. A non-transitory,computer-readable medium storing instructions that, when executed by oneor more processors, cause the one or more processors to: train, byanalyzing a test set of electronic communication documents, anartificial neural network to identify portions of conversation segments,within electronic communication documents, wherein the artificial neuralnetwork includes a plurality of layers, and wherein an input layer ofthe plurality of layers includes a plurality of input parameters eachcorresponding to a different characteristic of text-based content;receive a first electronic communication document that includes firsttext-based content; process the first text-based content of the firstelectronic communication document using the trained artificial neuralnetwork, wherein processing the first text-based content includesgenerating, within the plurality of layers, one or more positionindicators for the first electronic communication document, and whereinthe one or more position indicators include one or more segment portionindicators denoting positions of one or more portions of one or moreconversation segments within the first electronic communicationdocument; and determine an ordered relationship between the firstelectronic communication document and one or more other electroniccommunication documents using the one or more position indicators forthe first electronic communication document.
 16. The non-transitory,computer-readable medium of claim 15, wherein the instructions cause theone or more processors to train the artificial neural network at leastby comparing (i) one or more interim position indicators generated byanalyzing a first test document within the test set of electroniccommunication documents to (ii) one or more known position indicatorscorresponding to the first test document.
 17. The non-transitory,computer-readable medium of claim 15, wherein the plurality of inputparameters includes parameters indicative of one or more of (i) linebreak positioning, (ii) line spacing, (iii) character types, or (iv)character counts.
 18. The non-transitory, computer-readable medium ofclaim 15, wherein the artificial neural network is a recurrent neuralnetwork, and wherein the instructions cause the one or more processorsto train the artificial neural network at least by: determining weightsthat the artificial neural network will apply to one or both of (i) oneor more of the plurality of input parameters, and (ii) one or moreoutputs generated by one or more of the plurality of layers.
 19. Thenon-transitory, computer-readable medium of claim 15, wherein: theinstructions cause the one or more processors to train the artificialneural network further by training the artificial neural network toidentify one or more of (i) headers, (ii) message bodies, or (iii)signature blocks, within electronic communication documents; and the oneor more segment portion indicators include one or more segment sectionindicators denoting positions of one or more of (i) one or more headerswithin the first electronic communication document, (ii) one or moremessage bodies within the first electronic communication document, or(iii) one or more signature blocks within the first electroniccommunication document, and one or more field indicators denotingpositions of fields, within the first electronic communication document,that correspond to one or more field types.
 20. The non-transitory,computer-readable medium of claim 15, wherein the one or more positionindicators further include one or more segment indicators denotingpositions of one or more conversation segments within the firstelectronic communication document.